Systems and methods of determining dimensions of structures in medical images

ABSTRACT

Systems and methods for producing ultrasound images are disclosed herein. In one embodiment, ultrasound image data are acquired in discrete time increments at one or more positions relative to a subject. Control points are added by a user for two or more image frames and a processor interpolates the location of the control points for image frames obtained at in-between times.

RELATED PRIORITY APPLICATION

The present application claims the benefit of and priority, to U.S.Provisional Patent Application 62/335,624 filed May 12, 2016, which inherein incorporated by reference in its entirety.

PATENTS AND PATENT APPLICATIONS INCORPORATED BY REFERENCE

The following patents and publications are incorporated herein byreference in their entireties: U.S. Pat. No. 7,052,460, titled “SYSTEMFOR PRODUCING AN ULTRASOUND IMAGE USING LINE-BASED IMAGERECONSTRUCTION,” and filed Dec. 15, 2003; U.S. Pat. No. 7,255,648,titled “HIGH FREQUENCY, HIGH FRAME-RATE ULTRASOUND IMAGING SYSTEM,” andfiled Oct. 10, 2003; U.S. Pat. No. 7,901,358, titled “HIGH FREQUENCYARRAY ULTRASOUND SYSTEM,” and filed Nov. 2, 2006; U.S. Pat. No.8,317,714, titled “SYSTEMS AND METHODS FOR CAPTURE AND DISPLAY OF BLOODPRESSURE AND ULTRASOUND DATA,” and filed Nov. 27, 2012; and U.S. PatentPublication No. 2014/0128738, titled “Systems and methods for formingultrasound images,” and filed Nov. 5, 2013.

TECHNICAL FIELD

The present disclosure is generally directed to medical imaging systems.Embodiments of the present disclosure are directed to using ultrasoundimaging systems to determine boundaries and dimensions of anatomicalstructures in one or more ultrasound images.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an ultrasound imaging system configured inaccordance with embodiments of the disclosed technology.

FIG. 2 is a schematic view of an exemplary ultrasound image acquisitionin accordance with an embodiment of the disclosed technology.

FIG. 3A is a screenshot of a set of ultrasound frames acquired using anembodiment of the disclosed technology.

FIG. 3B is a 3D ultrasound image constructed in accordance with anembodiment of the disclosed technology.

FIG. 3C is a single two-dimensional ultrasound image frame used inconstructing the 3D ultrasound image of FIG. 3B.

FIGS. 4A and 4B are schematic diagrams illustrating user input of theboundaries of an anatomical structure.

FIGS. 5A and 5B are partially schematic diagrams illustratinginterpolation methods in accordance with an embodiment of the disclosedtechnology.

FIGS. 6A and 6B are partially schematic diagrams illustrating anotherinterpolation method in accordance with an embodiment of the disclosedtechnology.

FIG. 7 is a flowchart of a process of generating boundaries of one ormore anatomical structures in one or more ultrasound images inaccordance with an embodiment of the disclosed technology.

FIGS. 8A-8E are illustrations showing how the boundaries of anatomicalstructures are interpolated in accordance with some embodiments of thedisclosed technology.

FIG. 9 is a flowchart of a process of generating boundaries of one ormore anatomical structures in one or more ultrasound image frames inaccordance with an embodiment of the disclosed technology.

DETAILED DESCRIPTION

In ultrasound imaging devices, images of a subject are created bytransmitting one or more acoustic pulses into the body from atransducer. Reflected echo signals that are created in response to thepulses are detected by the same or a different transducer. The echosignals cause the transducer elements to produce electronic signals thatare analyzed by the ultrasound system in order to create a map of somecharacteristic of the echo signals such as their amplitude, power, phaseor frequency shift etc. The map can be used to form a two-dimension (2D)image.

Multiple 2D images formed using ultrasound echo signals received fromthe subject at different positions can be used to form athree-dimensional (3D) image of the subject. Several 3D images of thesubject acquired at different times and/or during different portions ofthe subject's cardiac or respiratory cycle can be used to form afour-dimensional (4D) image (e.g., a video and/or cineloop) of thesubject. An operator can use 3D and/or 4D image sets to determine avolume of a structure (e.g., a heart and/or another organ or structure)in the images. Operators may wish to measure, for example, a volume of aheart at a particular time point and/or multiple time points.Determining the volume of the heart typically involves tracing aboundary of a wall of the heart in each of several 2D images. The tracedboundaries can be used to form a 3D mesh describing the heart volume,and a dimension (e.g., a volume or surface area) of the heart can becalculated using the 3D mesh. For a 4D image set composed of several 3Dimages, however, tracing an outline of the structure in the individual2D image frames of each 3D image can be time consuming and tedious. If,for example, a 3D image includes 20 2D image frames, a 4D image madefrom 20 3D images, for example, the total data set can include 400individual image frames. Some prior art methods attempt to tracestructures in images automatically using segmentation, image analysisand/or other automatic tracing means. In many high frequency ultrasoundimages, however, border definitions of structures can be very unclearand thus automated analysis can be challenging and inaccurate. Anoperator with an understanding of the anatomy of an organ therefore maymore accurately discern where a boundary in an image should be drawn.

Embodiments of the disclosed technology can reduce the amount ofoperator input needed to determine boundaries of anatomical structuresin 3D and/or 4D ultrasound image sets, which can include dozens,hundreds or even thousands of images. In one embodiment, for example, amethod of operating an ultrasound imaging system to determine adimension of a region of interest in a subject includes acquiringultrasound echo data from the subject using a transducer coupled to theultrasound imaging system. The ultrasound echo data can be acquired at aplurality of times and at a plurality of positions relative to theregion of interest. The method further includes constructing, with theultrasound imaging system, a plurality of 3D images of the region ofinterest using the acquired ultrasound echo data. The individual 3Dimages can include a plurality of image frames and the individual imageframes can be acquired at one of the plurality of positions and at oneof the plurality of times. The ultrasound imaging system receives manualinput that can include, for example, user-selected points in a firstimage frame that define an anatomical boundary in the region ofinterest. The imaging system can compute an anatomical boundary in theregion of interest in a second image frame based on the user-selectedpoints in the first image frame. In some aspects, the first and secondimage frames include ultrasound data acquired at the same time butdifferent positions. In other aspects, however, the first frame includesdata acquired at the same position and a different time as the secondframe. The system can determine the dimension (e.g., a volume or asurface area) of the region of interest using the user-defined boundaryin the first image frame and the computed boundary in the second imageframe and can output the dimension of the region of interest to adisplay coupled to the ultrasound imaging system.

Suitable System

FIG. 1 is a block diagram illustrating an imaging system 100. The system100 operates on a subject 102. An ultrasound probe 112 proximate to thesubject 102 is configured to acquire image information. The ultrasoundprobe generates ultrasound energy at high frequencies, such as, but notlimited to, center frequencies between 15-60 MHz and higher. Further,ultrasound operating frequencies significantly greater than thosementioned above can be used. The subject 102 is connected toelectrocardiogram (ECG) electrodes 104 to obtain a cardiac rhythm fromthe subject 102. The electrodes 104 transmit the cardiac signal to anECG amplifier 106 to condition the signal for provision to an ultrasoundsystem 131. It is recognized that a signal processor or other suchdevice may be used instead of an ECG amplifier to condition the signal.If the cardiac signal from the electrodes 104 is suitable, then use ofan amplifier 106 or signal processor could be avoided entirely.

The ultrasound system 131 includes a control subsystem 127, an imageconstruction subsystem 129, sometimes referred to as a “scan converter”,a transmit subsystem 118, a receive subsystem 120 and a human-machineinterface 136 (e.g., a user interface and/or a user input). A processor134 is coupled to the control subsystem 127 and the display 116 iscoupled to the processor 134. A memory 121 is coupled to the processor134. The memory 121 can be any type of computer memory, and is typicallyreferred to as random access memory “RAM,” in which the software 123 isstored. The software 123 controls the acquisition, processing anddisplay of the ultrasound data allowing the ultrasound system 131 todisplay a high frame rate image so that movement of a rapidly movingstructure may be imaged. The software 123 comprises one or more modulesto acquire, process, and display data from the ultrasound system 131.The software comprises various modules of machine code, which coordinatethe ultrasound subsystems, as will be described below. Data is acquiredfrom the ultrasound system, processed to form complete images, and thendisplayed to the user on a display 116. The software 123 allows themanagement of multiple acquisition sessions and the saving and loadingof these sessions. Post processing of the ultrasound data is alsoenabled through the software 123.

The system for producing an ultrasound image using line-based imagereconstruction can be implemented using a combination of hardware andsoftware. The hardware implementation of the system for producing anultrasound image using line-based image reconstruction can include anyor a combination of the following technologies, which are all well knownin the art: discrete electronic components, discrete logic circuit(s)having logic gates for implementing logic functions upon data signals,an application specific integrated circuit having appropriate logicgates, a programmable gate array(s) (PGA), a field programmable gatearray (FPGA), one or more massively parallel processors, etc.

The software for the system for producing an ultrasound image usingline-based image reconstruction comprises an ordered listing ofexecutable instructions for implementing logical functions, and can beembodied in any computer readable medium for use by, or in connectionwith, an instruction execution system, apparatus, or device, such as acomputer-based system, processor-containing system, or other system thatcan fetch and execute the instructions.

In the context of this document, a “non-transitory computer-readablemedium” can be any physical means that can contain, store or transportthe program for use by or in connection with the instruction executionsystem, apparatus, or device. The non-transitory computer readablemedium can be, for example but not limited to, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatus,or device. More specific examples (a non-exhaustive list) of thenon-transitory computer-readable medium would include the following: anelectrical connection (electronic) having one or more wires, a portablecomputer diskette (magnetic), a random access memory (RAM), a read-onlymemory (ROM), an erasable programmable read-only memory (EPROM or Flashmemory) (magnetic), an optical fiber (optical), and a portable compactdisc read-only memory (CD-ROM) (optical).

The memory 121 can store the image data 110 obtained by the ultrasoundsystem 100. A non-transitory computer readable storage medium 138 iscoupled to the processor for providing instructions to the processor toinstruct and/or configure processor to perform steps or algorithmsrelated to the operation of the ultrasound system 131, as furtherexplained below.

The ultrasound system 131 can include a control subsystem 127 to directoperation of various components of the ultrasound system 131. Thecontrol subsystem 127 and related components may be provided as softwarefor instructing a general purpose processor or as specializedelectronics in a hardware implementation. The ultrasound system 131includes an image construction subsystem 129 for converting theelectrical signals generated by the received ultrasound echoes to datathat can be manipulated by the processor 134 and that can be renderedinto an image on the display 116. The control subsystem 127 is connectedto a transmit subsystem 118 to provide an ultrasound transmit signal tothe ultrasound probe 112. The ultrasound probe 112 in turn provides anultrasound receive signal to a receive subsystem 120. The receivesubsystem 120 also provides signals representative of the receivedsignals to the image construction subsystem 129. The receive subsystem120 is also connected to the control subsystem 127. The scan converteris directed by the control subsystem 127 to operate on the received datato render an image for display using the image data 110.

The ultrasound system 131 can include an ECG signal processor 108configured to receive signals from the ECG amplifier 106. The ECG signalprocessor 108 provides various signals to the control subsystem 127. Insome embodiments, the receive subsystem 120 also receives an ECG timestamp from the ECG signal processor 108. The receive subsystem 120 isconnected to the control subsystem 127 and an image constructionsubsystem 129. The image construction subsystem 129 is directed by thecontrol subsystem 127.

The ultrasound system 131 can further include a motor 180 (e.g., astepper motor, servo-torque motor, wobbler, etc.) configured to move theultrasound probe 112. The motor 180, for example, can be configured tomove the ultrasound probe 112 in one or more spatial directions (e.g.,along an x, y and/or z-axis) and/or rotate the ultrasound probe 112.

The ultrasound system 131 transmits and receives ultrasound data throughthe ultrasound probe 112, provides an interface to a user to control theoperational parameters of the imaging system 100, and processes dataappropriate to formulate still and moving images that represent anatomyand/or physiology. Images are presented to the user through theinterface display 116.

The human-machine interface 136 of the ultrasound system 131 takes inputfrom the user, and translates such input to control the operation of theultrasound probe 106. The human-machine interface 136 also presentsprocessed images and data to the user through the display 116.

The software 123 in cooperation with the image construction subsystem129 operate on the electrical signals developed by the receive subsystem120 to develop a high frame-rate ultrasound image that can be used toimage rapidly moving anatomy of the subject 102.

The control subsystem 127 coordinates the operation of the ultrasoundprobe 112, based on user selected parameters, and other system inputs.For example, the control subsystem 127 ensures that data are acquired ateach spatial location, and for each time window relative to the ECGsignal. Therefore, a full data set includes raw data for each timewindow along the ECG signal, and for each spatial portion of the imageframe. It is recognized that an incomplete data set may be used withappropriate interpolation between the values in the incomplete data setbeing used to approximate the complete data set.

The transmit subsystem 118 generates ultrasound pulses based on userselected parameters. The ultrasound pulses are sequenced appropriatelyby the control subsystem 127 and are applied to the probe 112 fortransmission toward the subject 102.

The receive subsystem 120 records the echo data returning from thesubject 102, and processes the ultrasound echo data based on userselected parameters. The receive subsystem 120 also receives a spatialregistration signal from the probe 112 and provides position and timinginformation related to the received data to the image constructionsubsystem 129.

Suitable Methods

FIG. 2 is a schematic view of an ultrasound image acquisition inaccordance with an embodiment of the disclosed technology. Theultrasound probe 112 transmits and receives ultrasound energy into aregion of interest 103 (e.g., a heart and/or another organ in asubject). The motor 180 moves the ultrasound probe 112 to each of aplurality of positions relative to the region of interest 103 that arespaced apart by a predetermined distance (e.g., 0.1 mm, 0.25 mm, 0.5mm). The ultrasound system 131 receives signals from the ultrasoundprobe 112 corresponding to the transmitted ultrasound energy and forms aplurality of two-dimensional (2D) ultrasound image frames or slices 250a-250 n of the region of interest 103. As described in more detail belowin reference to FIGS. 3A-3C, the ultrasound image frames 250 a-250 n canbe presented to the user at the interface 136 as a plurality of 2Dimages and/or can be used to form a three-dimensional (3D) image of theregion of interest 103.

FIG. 3A is a screenshot of an image set 360 comprising a plurality ofultrasound frames 350 acquired and constructed, for example, with theprobe 112 and the ultrasound system 131 (FIGS. 1 and 2). FIG. 3B is a 3Dultrasound image 365 constructed by the system 131 using one or more ofthe ultrasound frames 350 of FIG. 3A, including an ultrasound imageframe 350 a. FIG. 3C is an enlarged view of the ultrasound image frame350 a.

Referring to FIG. 3A, the plurality of ultrasound image frames 350 areformed using ultrasound data acquired at a plurality of positionsrelative to the region of interest of a subject (as shown, for example,in FIG. 2). The ultrasound system 131 presents the image set 360 to auser. The user can select one or more image frames 350 and inputinformation related to an edge, periphery or boundary of an anatomicalstructure (e.g., an organ such as a heart, liver, kidney, lung and/or aportion thereof) in at least one of the image frames 350. The user inputmay include manual input via a touchscreen, keyboard, mouse, touchpad,etc. FIG. 3A shows traced ultrasound image frames 350 a-k each includinga boundary 352 corresponding to edge or outline of an anatomicalstructure. As will be explained in further detail below in someembodiments of the disclosed technology, the ultrasound system 131receives user input related only to the boundary 352 in the image frames350 a and 350 k and the system generates boundaries in the interveningimage frames 350 b-j. In other embodiments, however, the system 131receives user input related to the boundary 352 in each of the imageframes 350 a-k.

Referring now to FIGS. 3A and 3C together, each boundary 352 includes aplurality of control points 354 that are input by the user. For example,the user might input 3-6 control points or in some cases more asrequired depending on the complexity of the anatomy being traced. Morecomplex shapes will require more user input. The ultrasound system 131connects adjacent control points 354 with a plurality of segments 356and in one embodiment, calculates an approximate center point 355 of thecontrol points 354. In the illustrated embodiment of FIG. 3C, thesegments 356 comprise cubic splines between adjacent control points 354.Allowing the user to draw or input the relatively few control points 354along the boundary 352 of an anatomical structure (e.g., a heart wall)and joining the control points with smoothly connected cubic splinesegments can significantly reduce the amount of time spent by the userdefining the boundaries of the anatomical structure in one or moreimages. Moreover, the cubic splines have a curve-like shape which can benaturally very consistent with curves along the anatomical structuressuch as, for example, a heart wall. In some embodiments, however, thesegments 356 may be linear and/or have shapes different from a cubicspline. In other embodiments, the system 131 may receive user input thatincludes an entire traced outline of the anatomical structure. Thesystem 131 can determine a dimension (e.g., a volume and/or a surfacearea) defined by the boundaries 352 in the image frames 350 a-k using,for example, software stored on the memory 121 (FIG. 1). Examples of oneor more techniques for determining a dimension of an anatomicalstructure can be found, for example, in U.S. Pat. No. 8,317,714incorporated by reference above.

FIGS. 4A and 4B are diagrams illustrating user input and boundarygeneration in accordance with an embodiment of the disclosed technology.Referring to FIG. 4A, an image set 460 includes a plurality of imageframes 450 a-h (e.g., the image frames 350 a-h of FIG. 3A) shownschematically without ultrasound data for clarity and ease ofunderstanding. The image frames 450 a-h include one or more so-called“key” frames 450 a and 450 h and several in-between frames 450 b-g. Inaccordance with one embodiment, a user inputs control points over one ormore anatomical structures shown in the key frame 450 a and an imagingsystem (e.g., the ultrasound system of FIG. 1) draws a boundary 452 a.The user repeats the sequence for the key frame 450 h and the imagingsystem draws a boundary 452 h. The system then generates a set ofboundaries 452 b-g in the in-between frames 450 b-g by interpolatingbetween the boundaries 452 a and 452 h.

In one embodiment, the user is not manipulating the underlyingultrasound data shown in each of the frames. Rather, the user isinputting a number of data points that define a shape that is separatefrom the underlying ultrasound data. The ultrasound system 131determines intermediate shapes between those that are input by the userand uses the input and determined shapes to calculate volumes, surfaceareas etc.

Referring to FIG. 4B, the user can modify the control points placed overan image frame 450 e by inputting additional control points or movingthe control points along a boundary 452 e′. The ultrasound systemtherefore computes a new boundary from the modified control points. Theultrasound system then performs a re-interpolation of the boundaries 452b-d for the in-between frame using the boundary 452 a and the modifiedboundary 452 e′ as well as the boundaries 452 f and 452 g. Using themodified boundary 452 e′ and the user input boundary 452 h, the accuracyof the boundaries in the in-between frames 452 b-d, 452 f and 452 g isincreased without user input in the in-between frames.

FIG. 5A is a diagram illustrating the generation of boundaries ofanatomical structures in accordance with an embodiment of the disclosedtechnology. FIG. 5A shows an image set 560 comprising key frames 550 aand 550 c over which a user has placed control points to defineboundaries 552 a and 552 c that trace a heart wall. As described abovein reference to FIGS. 3A-4B, the user inputs one or more control points554 (numbered in the frames 550 a-c as control points 554 a-c) over thekey frames 550 a and 550 c. A plurality of segments 556 (e.g., cubicsplines) connect adjacent control points 554 in the individual keyframes 550 a and 550 c. The system (e.g., the system 131 of FIG. 1) canautomatically generate control points 554 b for the in-between frame 550b by interpolation and/or morphing along lines 559 to define the controlpoints for the in-between frames.

As in traditional key frame animation, a bounding pair of frames definethe key frames. The interior, or ‘in-between’ frames may only includeonly a slight modification of the outer frames. For example, a heartwall boundary on the in-between frames may be sufficiently approximatedusing the information present in the key frames. The in-between frametraces can be morphed based on their proximity to the traces defined bythe user. In some embodiments, all the walls of the heart may besuccessfully traced from control points placed over only a few ‘key’frames drawn by the user. The traces on the other frames can be morphedor interpolated representations of the boundaries in these key frames.As the user adjusts the control points on the key frames, the controlpoints for some or all of the in-between frames may be automaticallyadjusted based on the updated information. As discussed above, the usercan adjust control points in the in-between frames that he or she deemsnot lying on or near a boundary of the heart wall. This additionalinformation is then applied to the entire data set to improve thequality of the remaining in-between frames.

FIG. 5B is a diagram illustrating the generation of boundaries ofanatomical structures in accordance with another embodiment of thedisclosed technology. An image set 561 includes key frames 550 d and 550f having corresponding control points placed over the image frame thatdefine boundaries 552 d and 552 f that represent a heart wall. In theillustrated embodiment, the boundaries 552 d and 552 f havesubstantially the same shape but different sizes. Rather than the userinputting a completely new set of control points over an in-betweenframe, the system can automatically generate control points 554 e byinterpolating between control points 554 d in the key frame 550 d andcontrol points 554 f in the key frame 550 f. In some instances, forexample, it is faster to resize or move the entire boundary. The systemcan copy the control points and connecting splines for the boundaryshape from either of the key frames and allow the user to simply enlargeor contract the boundary size for the in-between frame without changingits shape. In some embodiments, the boundary can also be rotated and/orshifted to a new position without changing its size or shape.

FIGS. 6A and 6B illustrate another interpolation method in accordancewith an embodiment of the disclosed technology. Referring to FIGS. 6Aand 6B together, the system generates a number (e.g., 128, 256, 512) ofpoints 658 along the splines 556 between adjacent control points 554.User interaction is simplified by only presenting and allowing usermodification of control points. For example, the original boundarypoints as entered by the user are control points. The user may enter forexample, 3,4,5 or any number of control points. The system thengenerates internal points 658 which are not presented to the user butused for internal calculations. In some embodiments, the indexing of thepoints 658 can start at the same rotational position along each of thesplines 556. For example, the system can begin ordering with the point658 at a 12 o'clock position (i.e., a 0° and/or vertical position asshown in the diagram) and continue to index additional points 658 in aclockwise and/or counterclockwise direction. The system can then selecta matching pair of points from the left key frame 550 d and the rightkey frame 550 f (FIG. 6B). For example, from each point set 658 and 658′select the n'th point (i.e. the 16′th point). As shown in FIG. 6B, thepoint 658 in frame 550 d is matched with a point 658′ in frame 550 f.Given a coordinate location of each point (horizontal and vertical, x,y, defined in mm), a parametric linear equation can be calculatedbetween these matched points 658 and 658′. Using these equations and theposition of the image frame 550 e, the system selects a coordinatelocation for new point 658″ on the in-between frame 550 e. In someembodiments, position might be time, or frames. This is repeated for allof the calculated internal point pairs (for example, 128, 256, 512 pointpairs). The result is a boundary representation 658″ on frame 550 e.

In some embodiments, a linear equation, e.g. y=mx+b, is used tocalculate the points on the in-between frame 550 e where the dependentvariable x is frame position or time. For example, for each of the twospatial parameters of a point (horizontal and vertical position) alinear equation defined as y=mx+b can be used to determine correspondingpoints for the in-between frame 550 e using the values from the keyframes 550 d and 550 f. In this equation, y is one of the spatialparameters (e.g., position) and x is the frame position, which can bemeasured, e.g., in units of frames (or time or position). To derive thephysical position of the point for the in-between frame 550 e, thelinear equation defined can be used by inserting the variable x for thecorrect frame position. For example, referring to FIG. 6B, if controlpoints are interpolated within a time point from control points 658 to658′, these points might be described with coordinates x, y, z where xis the horizontal position within the 2D image 550 d, y is the verticalposition within the 2D image 550 d, and z describes the position of theimage as acquired by the 3D motor. For example, control point 658 mightbe x, y, z (5.5 mm, 3.3 mm, 6 mm). Control point 658′ might be (4.0 mm,5.2 mm, and 8 mm). To interpolate the location of the control point 658″on frame 550 e where the z position of this frame is 7 mm, the linearequations are solved for the two parameters as follows y=mz+b andx=mz+b. Thus the equation for x values is x=−0.75z+10.0 and the equationfor y values is y=0.95 z−2.4, both as a function of z. Thus the point658″ is (4.75 mm, 4.25 mm, 7 mm). When interpolating across time points,then the z axis becomes time and the two points would be described x, y,t where t is in units of time. For example, the two points might be 658(5.5 mm, 3.3 mm, 6 ms) and 658′ (4.0 mm, 5.2 mm, and 8 ms). The linearequations are solved for the two parameters as follows y=mt+b andx=mt+b. Thus the equation for x values is x=−0.75t+10.0 and the equationfor y values is y=0.95 t−2.4, both as a function of t. Thus, theinterpolated control point 658″ has values (4.75 mm, 4.25 mm, 7 ms). Theprocess above repeats for all the point pairs from the key frames 550 dand 550 f. The result is that the points 658″ for the in-between frame550 e will have a shape defined by the new points as shown in FIG. 6B.

In some embodiments, the internal points along the splines between theuser defined control points can be interpolated for the in-betweenframes instead of using the user defined control points. Each frame mayinclude several hundred points (e.g. 128, 256, 512 etc.) calculatedalong the splines. In one embodiment, each of the these points isnumbered starting from a common position such as the 12 o'clockposition. A mathematical line can be determined between all, or fewerthan all, of these points on one key frame and the same numbered pointon a second key frame to calculate a corresponding point in thein-between frame.

In order to reduce the number of interpolated points that are shown tothe user for the in-between frame, the system determines which of theinterpolated points corresponds most closely to the user determinedcontrol points in the key frames. For example, a user defined controlpoint in one key frame might be closest to point 63 in one key framewhile the corresponding user defined control point in the other keyframe might be closest to point 75. If the number of control points inkey frame 550 d does not match the number in key frame 550 f, additionalcontrol points can be calculated and inserted along at least one of thesplines 658 or 658′ such that they each contain the same number ofcontrol points. This new control point only exists for internalcalculations and not be shown to the user. The same or similar linearinterpolation described above can be used to calculate positions for anew control point index for the in-between frame 550 e. For example, thelinear equation for the system {63, 6 mm} and {75, 8 mm} is solved forin-between frame 550 e, which exists at 7 mm. The equation of thissystem is Index=6x+27 where x is the position of the frame. Calculatingfor frame 550 e at 7 mm, the index is thus 69. Therefore, interpolatedpoint 69 is selected as a control point to show to the user for thein-between frame. This process is repeated for each pair of controlpoints on key frames 550 d and 550 f. The result is a set ofinterpolated control points on frame 550 e which are presented to theuser and can be selected and modified.

In some embodiments, instead of using linear interpolation of points andcontrol point indexes between two frames, cubic interpolation or evenquadratic interpolation could be used. In this case, instead of usingtwo bounding frames to solve the system three frames would be used inthe case of cubic interpolation and four in the case of quadraticinterpolation. Cubic spline interpolation could also be used which woulduse data from all the frames to generate data for the in-between frames.

Once the interpolated points are computed, they can be plotted on thein-between frame and splines calculated that connect the interpolatedcontrol points. The user can move the position of the calculated controlpoints. If the position a control point is moved to better coincide withan anatomical feature, the in-between frame can then be designated as akey frame and the positions of the control points in the frame can beused as a basis for determining the positions of control points in otherin-between frames. In some instances, for example, it is faster toresize or move the entire boundary, rather than modify individualcontrol points. The user can enlarge or contract the boundary size forthe in-between frame without changing its shape. In some embodiments,the boundary can also be rotated and/or shifted to a new positionwithout changing its size or shape.

FIG. 7 is a flowchart illustrating a process 700 of generating aboundary of one or more anatomical structures (e.g., heart, liver,kidney and/or one or more portions thereof) in one or more 2D ultrasoundimage frames. In some embodiments, instructions for causing a processorto implement the process 700 can be stored on a memory (e.g., the memory121 of FIG. 1) and executed by the processor (e.g., the processor 134 ofFIG. 1) of an ultrasound imaging system (e.g., the system 131 of FIG.1). At block 710, the process 700 generates and transmits ultrasoundenergy (e.g., ultrasound energy having a center frequency greater thanabout 15 MHz) from an ultrasound transducer probe (e.g., the probe 112of FIGS. 1 and 2) toward a region of interest (e.g., a heart, liver,kidney) in a subject (e.g., a human or an animal, such as a rat or amouse). At block 720, the process 700 acquires ultrasound datacorresponding to ultrasound echoes received from the subject and usesthe acquired ultrasound data to form one or more ultrasound image frames(e.g., the image frames 350 of FIG. 3A). In some embodiments, theprocess 700 can acquire the image data at discrete positions relative tothe region of interest. As described above, for example, in reference toFIG. 2, the process 700 can control a motor (e.g., the motor 180) andmove the probe a predetermined incremental distances relative to theregion of interest to acquire ultrasound data at a plurality ofpositions. In other embodiments, however, the process 700 acquires theultrasound image data from the region of interest in a single dataacquisition.

A block 730, the process 700 optionally constructs one or more 3D or 4Dimages using the image frames acquired at block 720. The process 700 canform a 3D image using a plurality of 2D image frames acquired atpredetermined positions relative to the region of interest. The process700 can also form one or more 4D images using, for example, several 3Dimages acquired at different portions of the subject cardiac cycle. Inone embodiment, the process 700 constructs a 4D image using one or moremethods disclosed in the applicant's co-pending application Ser. No.14/072,755, published as U.S. Patent Publication No. 2014/0128738, andincorporated by reference above. In other embodiments, the processproceeds directly to block 740 without constructing 3D and/or 4D images.

At block 740, the process 700 presents the 2D image frames acquired atblock 720 to an operator (e.g., the image set 360 of FIG. 3A). Theoperator selects one or more 2D image frames as key frames and placesone or more points or markers as control points near and/or along aboundary of an anatomical structure in the individual image frames.

At block 750, the process 700 generates a boundary in one or moreacquired image frames based on the user input received at block 740. Asdiscussed above in reference to FIGS. 4A and 4B, for example, the usercan select two key frames and trace or plot points around an anatomicalboundary in the key frames. Splines or other mathematical curves/linescan be computed to connect the control points to trace the perimeter ofthe anatomical features. The process 700 can interpolate betweenindividual control points in a first key frame and corresponding controlpoints in a second key frame to automatically generate boundaries in oneor more in-between image frames that are between the key frames.

At block 760, the process can present the 2D image frames with drawnboundaries to the user via a display for additional editing. Asdiscussed above in reference in FIG. 4B, for example, the process 700can receive user input for any of the generated boundaries to furtherincrease accuracy of the boundaries. For example, the user can adjust aboundary generated in one of the in-between frames in which thegenerated boundary does not match up with a boundary of the anatomicalstructure in the image. Each in-between image that the user manuallyadjusts becomes a new key frame and therefore can improve the accuracyof the remaining in-between frames.

At block 770, the process 700 can determine a measurement (e.g., asurface area, circumference, volume) of a region defined by themanually-input and generated boundaries in the 2D image frames.

At block 780, the process 700 outputs the determined measurement(s) tothe operator.

FIGS. 8A-8E are schematic diagrams illustrating the generation ofboundaries of anatomical structures in a 4D image set 860 in accordancewith some embodiments of the disclosed technology. In the figures,anatomy boundaries shown in solid lines are input by a user, while thoseshown in dashed lines are computed. Referring to FIGS. 8A-8E together, a4D image set includes a plurality of 2D image frames of a region ofinterest arranged in a grid of rows and columns. The rows A-M represent2D image slices taken at different positions relative to the region ofinterest as discussed above, for example, with reference to FIG. 2. Thecolumns TP1-TP9 represent 2D images taken at different time pointsand/or periods of time. Taken together, the 2D image frames in eachcolumn can comprise a 3D image of the region of interest acquired at aparticular time TP1-TP9. In some embodiments, the times TP1-TP9correspond to different portions of the subject's cardiac cycle. TP1 canrepresent, for example, a first time point in the subject's heart cycle(0 ms time). TP5 can represent, for example, a time point approximately½ way through the heart cycle. Depending on the heart rate, TP5 might be50 ms (if the period of the heart cycle is 100 ms, for example). In someaspects, TP9 and TP1 can generally represent the same or similar timepoint in a cycle as the heart completes one cycle and begins asubsequent cycle. In some aspects, the data obtained at time points atthe 1/4 point (TP3) and the ¾ point (TP7) are similar.

In the illustrated embodiment, the 4D image set includes 2D image framesacquired at 13 positions and 9 time points. In some embodiments,however, the 4D image set can include 2D image frames acquired at feweror more positions and/or time points. Moreover, the individual 2D imageframes are shown for illustrated purposes as two-dimensional grid. Inother embodiments, however, the 2D frames can be presented in anysuitable format. For example, in some embodiments, the ultrasound systemmay present one image or a portion of the images at a time.

Referring now to FIG. 8A, an operator inputs points along a boundary ofan anatomical structure in an image frame 850 a and 850 b. Theultrasound system (e.g., the system 131 of FIG. 1) draws correspondingboundaries 852 a and 852 b in the image on TP1 on frames 850 a (TP1-A)and 850 b (TP1-M), respectively. The ultrasound system then generatescontrol points for the in-between frames TP1-B through TP1-L. Theoperator then modifies the position of any of the control points anyin-between frames in accordance with the methods described above suchthat all the boundaries for this time point (TP1) suitably match theanatomical structure. For example, a user may modify the location of thecontrol points in frame TP1-G to better coincide with the boundary ofthe anatomy captured in the ultrasound image. In some embodiments, theboundary may not start on TP1-A and finish on TP1-M but start and stopon other positions (e.g., the operator might begin placing controlpoints on the image frame TP1-C and finish on TP1-K depending on theextents of the anatomical structure being traced).

Referring now to FIG. 8B, the system then directs the user to drawcompleted boundaries for at least two image frames obtained at twoadditional time points; the ¼ position time point TP3 and the ½ positiontime point TP5. In other embodiments the ¼ and ½ time points will bedifferent depending on how many total time points were used inperforming a scan. The operator completes similar steps as describedabove to complete the trace for those time points. In some embodiments,the system automatically readjusts the display such the operator ispresented only with image frames obtained at the ¼ time point fortracing. Once the image frames at the ¼ time points are complete, thesystem then moves to the ½ time point and instructs the user to completethe boundaries on at least two image frames obtained at this time point.After this has been completed, the user will have drawn completeboundaries for three time points TP1, TP3 (e.g. the ¼ position) and TP5(e.g. the ½ position). The system will generate control point locationsfor the in-between frames based on those time points. For example,referring to FIG. 8B., TP3 D,E,F,H,I,J and TP5 E,F,H,I. The operator isnot required to start the boundary trace on the first position (e.g.position A) and complete it on the last position (e.g. position M) butcan start and stop on any two positions. If the anatomical structure isa heart, for example, it will compress in the middle of the cardiaccycle.

Referring now to FIG. 8C, as discussed above, for example, TP1 and TP9occur when the heart is at a similar position in the subject's cardiaccycle. This means the boundary shape will also be similar. Also, dataobtained at TP3 and TP7, referencing the ¼ and ¾ positions points in thesubject's cardiac cycle will generally also result in similar boundaryshapes. Accordingly, the boundary of the anatomical structure at similarportions of a subject's cardiac cycle can have the same or similar sizeand shape. In one embodiment, the system copies the boundaries from timeTP1 to time TP9 and from TP3 to time TP7. In other embodiments, however,the operator may manually input the boundaries on frames obtained attimes TP7 and TP9. The system will generate the data for the in-betweenframes for these time points.

Referring now to FIG. 8D, the system then generates boundaries for allremaining time points which have not been either directly drawn by theoperator (TP1, TP3, and TP5) or copied by the system from these existingboundaries (TP7 and TP9). These time points are referred to as “fixedtime points” while the data for time points TP2, TP4, TP6, TP8 aregenerated by the system. These time points are referred to as“in-between time points”. In some embodiments, depending on the numberof total time points acquired, more in-between time points may exist. Togenerate a boundary for any frame obtained at an in-between time point,the system looks “horizontally” across different time points andinterpolates data from two bounding frames obtained at the “fixed timepoints”. For example, to generate a boundary for a frame TP2-F, thesystem interpolates from fixed frames TP1-F and TP3-F. This process inrepeated for all frames on all in-between time points.

In some embodiments, the anatomical structure being traced, does notextend to the first and last acquired position for all time points. Aswith cardiac motion, the heart compresses as it nears the middle of thecardiac cycle. Thus the boundaries may only exist on a portion of thetime point frames (e.g. in FIG. 8D, on TP5, the boundaries are drawn onframes TP5-D through TP5-J). When the system generates the boundariesfor the in-between time points, a decision is made if a boundary existson each frame based on the existence of bounding fixed frames. Referringto FIG. 8E, by joining the exterior bounding frames for each time point,a region mask can be formed differentiating which frames containboundaries that are to be drawn and which do not. Frames outside thisregion (e.g. those drawn in white) do not receive a system generatedin-between boundary. This region mask will change as the operator makesadjustments.

In some embodiments, image recognition software can analyze the 2D imageframes and only display to the user those image frames for positions atwhich the anatomy can be seen. This can help the user enter controlpoints for image frames that are used by the system to computeinterpolated control points. Image frames at positions in which theanatomy does not extend can be hid from the user in order to reduce thenumber of image frames that the user has to view while inputting thecontrol points.

The operator may now make adjustments to all in-between and fixedboundaries. Changes will result in updates to all in-between boundaries.For example, if the operator adjusts the fixed boundary in frame TP5-G(FIG. 8C), this would result in the boundaries of the borderingin-between frames of this time point being automatically updated by thesystem (e.g. TP5-E, TP5-F, TP5-H, TP5-I). Since this results in a changeto position F, the other in-between time points would also be updated(e.g. TP4-F and TP2-F, TP6-F, TP8-F). As one can appreciate, this couldresult in global changes throughout all in-between frames. In oneembodiment, to ensure the changes do not result in infinitely recursiveupdates, once a time point has been updated by the user (any frame onthe time point) it will only update the boundaries of the frames in thattime point “vertically”. For example, in between frame TP5-E will onlybe interpolated based on frames in TP5 (e.g. TP5-D and TP5-G). Also,in-between time points (e.g. TP4) would only be interpolated“horizontally”. For example, in-between frame TP4-E will only beinterpolated based on frames in TP3 and TP5 (e.g. TP3-E and TP5-E). If auser makes an adjustment to any boundary on TP4, it ceases to be anin-between time point and becomes a fixed time point and is no longerinterpolated “horizontally”.

FIG. 9 is a flowchart of a process 900 of generating boundaries of oneor more anatomical structures in one or more 3D ultrasound images inaccordance with an embodiment of the disclosed technology. At block 990,a user is prompted to trace the boundaries of an anatomical structure ina frame at the a first point in a cardiac cycle, at a second point thatis generally a ¼ cycle later, and a third point that generally a halfcycle later. At block 992, the process 900 copies the boundaries enteredonto the first and 1/4 image frame to a frame for an nth later timepoint that is approximately the same or a similar portion of thesubject's cardiac cycle (e.g., time point TP1 to TP9 in FIGS. 8A-8E, andTP3 to TP7).

At block 994, the process 900 automatically generates boundaries inimage frames acquired at a time after the first time point and themid-cycle frame. Computed data can then be copied into frames atcorresponding time periods in the cardiac cycle.

At block 996, the process 900 receives input indicative of an adjustmentin the position of the control points one or more of the image framesgenerated at block 994. At block 998, the process 900 adjusts theboundaries in the remaining generated frames based on the adjustments atblock 996. In some embodiments, the process 900 only adjusts framesacquired at the same position as the adjusted frames in block 996. Inother embodiments, however, the process 900 adjusts all of the generatedboundaries based on each boundary adjusted by the operator.

Conclusion

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense, as opposed to anexclusive or exhaustive sense; that is to say, in the sense of“including, but not limited to.” As used herein, the terms “connected,”“coupled,” or any variant thereof means any connection or coupling,either direct or indirect, between two or more elements; the coupling orconnection between the elements can be physical, logical, or acombination thereof. Additionally, the words “herein,” “above,” “below,”and words of similar import, when used in this application, refer tothis application as a whole and not to any particular portions of thisapplication. Where the context permits, words in the above DetailedDescription using the singular or plural number may also include theplural or singular number respectively. The word “or,” in reference to alist of two or more items, covers all of the following interpretationsof the word: any of the items in the list, all of the items in the list,and any combination of the items in the list.

The above Detailed Description of examples of the disclosed technologyis not intended to be exhaustive or to limit the disclosed technology tothe precise form disclosed above. While specific examples for thedisclosed technology are described above for illustrative purposes,various equivalent modifications are possible within the scope of thedisclosed technology, as those skilled in the relevant art willrecognize. For example, while processes or blocks are presented in agiven order, alternative implementations may perform routines havingsteps, or employ systems having blocks, in a different order, and someprocesses or blocks may be deleted, moved, added, subdivided, combined,and/or modified to provide alternative or sub combinations. Each ofthese processes or blocks may be implemented in a variety of differentways. Also, while processes or blocks are at times shown as beingperformed in series, these processes or blocks may instead be performedor implemented in parallel, or may be performed at different times.Further any specific numbers noted herein are only examples: alternativeimplementations may employ differing values or ranges.

The teachings of the disclosed technology provided herein can be appliedto other systems, not necessarily the system described above. Theelements and acts of the various examples described above can becombined to provide further implementations of the disclosed technology.Some alternative implementations of the disclosed technology may includenot only additional elements to those implementations noted above, butalso may include fewer elements.

These and other changes can be made to the disclosed technology in lightof the above Detailed Description. While the above description describescertain examples of the disclosed technology, and describes the bestmode contemplated, no matter how detailed the above appears in text, thedisclosed technology can be practiced in many ways. Details of thesystem may vary considerably in its specific implementation, while stillbeing encompassed by the disclosed technology disclosed herein. As notedabove, particular terminology used when describing certain features oraspects of the disclosed technology should not be taken to imply thatthe terminology is being redefined herein to be restricted to anyspecific characteristics, features, or aspects of the disclosedtechnology with which that terminology is associated. In general, theterms used in the following claims should not be construed to limit thedisclosed technology to the specific examples disclosed in thespecification, unless the above Detailed Description section explicitlydefines such terms.

1. A method of operating an ultrasound imaging system to determine adimension of an anatomical structure in a subject, comprising:transmitting ultrasound energy from a transducer coupled to theultrasound imaging system into the subject; acquiring ultrasound echodata from the subject using the transducer; displaying, with theultrasound imaging system, a plurality of two-dimensional (2D) imageframes of a region of interest in the subject using the acquiredultrasound echo data; receiving user input at a user interface of theultrasound imaging system, wherein the user input includes user-selectedcontrol points in at least a first and second image frame definingboundaries of an anatomical at different points in time; andinterpolating the user-selected control points to determine boundariesof the anatomy in image frames that are obtained at times between thefirst and second image frames.
 2. The method of claim 1, furthercomprising computing a boundary of the anatomical structure from theuser supplied control points and copying the boundary from an imageframe obtained at one point in a cardiac cycle to an image frameobtained at a similar point the cardiac cycle.
 3. The method of claim 1further comprising: displaying, with the ultrasound imaging system, aplurality of two-dimensional (2D) image frames of a region of interestin the subject using the acquired ultrasound echo data; receiving userinput at a user interface of the ultrasound imaging system, wherein theuser input includes user-selected control points in at least a first andsecond image frame defining boundaries of the anatomical structure atdifferent positions; and interpolating the user-selected control pointsto determine boundaries of the anatomical structure in image frames thatare obtained at different positions.
 4. The method of claim 3, furthercomprising displaying to a user those 2D ultrasound frames at positionsinto which the anatomical structure extends at various portions of acardiac cycle.
 5. A method of operating an ultrasound imaging system todetermine a volume of a region of interest in a subject, comprising:acquiring ultrasound echo data from the subject using a transducercoupled to the ultrasound imaging system, wherein the ultrasound echodata is acquired at a plurality of times and at a plurality of positionsrelative to the region of interest; constructing, with the ultrasoundimaging system, a plurality of three-dimensional (3D) images of theregion of interest in the subject using the acquired ultrasound echodata, wherein the individual 3D images comprise a plurality of imageframes and wherein the individual image frames are acquired at one ofthe plurality of positions and at one of the plurality of times;receiving manual input at a user interface of the ultrasound imagingsystem, wherein the manual input includes user-selected points in afirst image frame of the plurality of image frames along a user-definedboundary of the region of interest in the first image frame, and whereinthe first image frame includes ultrasound echo data acquired at a firstposition and a first time; and generating a boundary of the region ofinterest in at least a second image frame based on the user-selectedpoints in the first image frame, wherein the second image frame includesultrasound echo data acquired at a second position and a second time. 6.The method of claim 5, further comprising: receiving user-selectedpoints in a third image frame of the plurality of image frames that forma boundary of the region of interest in the third image frame, whereinthe third image frames includes ultrasound echo data acquired at a thirdposition and a third time, and wherein the second time is between thefirst and third times.
 7. The method of claim 6 wherein generating theboundary of the region of interest in the second image frame furthercomprises: interpolating the user-selected points in the first and thirdimage frames; and connecting the adjacent ones of interpolated points inthe second image frame with a cubic spline.
 8. The method of claim 7wherein the interpolating is weighted based on the distances between thesecond position and the first and third positions, respectively.
 9. Themethod of claim 7 wherein the interpolating is weighted based on thedifferences in times between the second time and the first and thirdtimes, respectively.
 10. The method of claim 5 wherein the ultrasoundenergy has a center frequency greater than or equal to about 20 MHz. 11.The method of claim 5 wherein the individual 3D ultrasound images arerepresentative of corresponding different portions a heart during asubject's cardiac cycle.
 12. An ultrasound imaging system to determine adimension of an anatomical structure in a subject, comprising: anultrasound transducer configured to transmit ultrasound energy into asubject and acquire ultrasound echo data from the subject; a processorconfigured to: display a plurality of two-dimensional (2D) image framesof a region of interest in the subject using the acquired ultrasoundecho data; receive user input of user-selected control points for atleast a first and second 2D image frame defining boundaries of ananatomical structure at different points in time; and interpolate theuser-selected control points to define boundaries of the anatomicalstructure for image frames that are obtained at times between the firstand second image frames.
 13. The system of claim 12, wherein theanatomical structure varies in size over a cardiac cycle and theprocessor is configured to display a number of 2D image frames in whicha maximum and minimum size of the anatomical structure can be seen atvarious positions in the region of interest for a time during thecardiac cycle.